Face Generation

In this project, you'll define and train a DCGAN on a dataset of faces. Your goal is to get a generator network to generate new images of faces that look as realistic as possible!

The project will be broken down into a series of tasks from loading in data to defining and training adversarial networks. At the end of the notebook, you'll be able to visualize the results of your trained Generator to see how it performs; your generated samples should look like fairly realistic faces with small amounts of noise.

Get the Data

You'll be using the CelebFaces Attributes Dataset (CelebA) to train your adversarial networks.

This dataset is more complex than the number datasets (like MNIST or SVHN) you've been working with, and so, you should prepare to define deeper networks and train them for a longer time to get good results. It is suggested that you utilize a GPU for training.

Pre-processed Data

Since the project's main focus is on building the GANs, we've done some of the pre-processing for you. Each of the CelebA images has been cropped to remove parts of the image that don't include a face, then resized down to 64x64x3 NumPy images. Some sample data is show below.

If you are working locally, you can download this data by clicking here

This is a zip file that you'll need to extract in the home directory of this notebook for further loading and processing. After extracting the data, you should be left with a directory of data processed_celeba_small/

# can comment out after executing
#!unzip processed_celeba_small.zip

'unzip' is not recognized as an internal or external command,
operable program or batch file.

data_dir = 'processed_celeba_small/'

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import pickle as pkl
import matplotlib.pyplot as plt
import numpy as np
import problem_unittests as tests
#import helper

%matplotlib inline

Visualize the CelebA Data

The CelebA dataset contains over 200,000 celebrity images with annotations. Since you're going to be generating faces, you won't need the annotations, you'll only need the images. Note that these are color images with 3 color channels (RGB)#RGB_Images) each.

Pre-process and Load the Data

Since the project's main focus is on building the GANs, we've done some of the pre-processing for you. Each of the CelebA images has been cropped to remove parts of the image that don't include a face, then resized down to 64x64x3 NumPy images. This pre-processed dataset is a smaller subset of the very large CelebA data.

There are a few other steps that you'll need to transform this data and create a DataLoader.

Exercise: Complete the following get_dataloader function, such that it satisfies these requirements:

• Your images should be square, Tensor images of size image_size x image_size in the x and y dimension.
• Your function should return a DataLoader that shuffles and batches these Tensor images.

ImageFolder

To create a dataset given a directory of images, it's recommended that you use PyTorch's ImageFolder wrapper, with a root directory processed_celeba_small/ and data transformation passed in.

# necessary imports
import torch
from torchvision import datasets
from torchvision import transforms

def get_dataloader(batch_size, image_size, data_dir='processed_celeba_small/'):
    """
    Batch the neural network data using DataLoader
    :param batch_size: The size of each batch; the number of images in a batch
    :param img_size: The square size of the image data (x, y)
    :param data_dir: Directory where image data is located
    :return: DataLoader with batched data
    """
    
    # TODO: Implement function and return a dataloader
    
    transform_train = transforms.Compose([transforms.Resize(image_size),
                                          transforms.ToTensor()])
    image_dataset = datasets.ImageFolder(root=data_dir, transform=transform_train)
    n_workers = 0
    train_loader = torch.utils.data.DataLoader(dataset=image_dataset,
                                        batch_size=batch_size,
                                        shuffle=True,
                                        num_workers=n_workers)
    
    return train_loader

Create a DataLoader

Exercise: Create a DataLoader celeba_train_loader with appropriate hyperparameters.

Call the above function and create a dataloader to view images.

• You can decide on any reasonable batch_size parameter
• Your image_size must be 32. Resizing the data to a smaller size will make for faster training, while still creating convincing images of faces!

# Define function hyperparameters
batch_size = 32
img_size = 32

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
# Call your function and get a dataloader
celeba_train_loader = get_dataloader(batch_size, img_size)

Next, you can view some images! You should seen square images of somewhat-centered faces.

Note: You'll need to convert the Tensor images into a NumPy type and transpose the dimensions to correctly display an image, suggested imshow code is below, but it may not be perfect.

# helper display function
def imshow(img):
    npimg = img.numpy()
    plt.imshow(np.transpose(npimg, (1, 2, 0)))

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
# obtain one batch of training images
dataiter = iter(celeba_train_loader)
images, _ = dataiter.next() # _ for no labels

# plot the images in the batch, along with the corresponding labels
fig = plt.figure(figsize=(20, 4))
plot_size=20
for idx in np.arange(plot_size):
    ax = fig.add_subplot(2, plot_size/2, idx+1, xticks=[], yticks=[])
    imshow(images[idx])

<ipython-input-5-13b4d9c7a4be>:17: MatplotlibDeprecationWarning: Passing non-integers as three-element position specification is deprecated since 3.3 and will be removed two minor releases later.
  ax = fig.add_subplot(2, plot_size/2, idx+1, xticks=[], yticks=[])

Exercise: Pre-process your image data and scale it to a pixel range of -1 to 1

You need to do a bit of pre-processing; you know that the output of a tanh activated generator will contain pixel values in a range from -1 to 1, and so, we need to rescale our training images to a range of -1 to 1. (Right now, they are in a range from 0-1.)

# TODO: Complete the scale function
def scale(x, feature_range=(-1, 1)):
    ''' Scale takes in an image x and returns that image, scaled
       with a feature_range of pixel values from -1 to 1. 
       This function assumes that the input x is already scaled from 0-1.'''
    # assume x is scaled to (0, 1)
    # scale to feature_range and return scaled x
    min, max = feature_range
        
    return x*(max - min) + min

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
# check scaled range
# should be close to -1 to 1
img = images[0]
scaled_img = scale(img)

print('Min: ', scaled_img.min())
print('Max: ', scaled_img.max())

Min:  tensor(-1.)
Max:  tensor(0.8196)

---

Define the Model

A GAN is comprised of two adversarial networks, a discriminator and a generator.

Discriminator

Your first task will be to define the discriminator. This is a convolutional classifier like you've built before, only without any maxpooling layers. To deal with this complex data, it's suggested you use a deep network with normalization. You are also allowed to create any helper functions that may be useful.

Exercise: Complete the Discriminator class

• The inputs to the discriminator are 32x32x3 tensor images
• The output should be a single value that will indicate whether a given image is real or fake

import torch.nn as nn
import torch.nn.functional as F

# helper function creating a convolutional layer
def conv(n_inputs, n_outputs, kernel_size, stride=2, padding=1, batch_norm=True):
    """Create convolutional layer with optional batch normalization."""
    layers = []
    conv_layer = nn.Conv2d(n_inputs, n_outputs, kernel_size, stride, padding, bias=False)
    
    layers.append(conv_layer)
    
    if batch_norm:
        # add batch normalization layer
        layers.append(nn.BatchNorm2d(n_outputs))
    
    return nn.Sequential(*layers)

class Discriminator(nn.Module):

    def __init__(self, conv_dim):
        """
        Initialize the Discriminator Module
        :param conv_dim: The depth of the first convolutional layer
        """
        super(Discriminator, self).__init__()
        
        self.conv_dim = conv_dim
        
        # Output_size = (Input_size - 1) * Strides + Kernel_Size - (2 *padding) + Output Padding

        # complete init function
        # 32x32x3 input of RGB image
        # no batch normalization on the first layer as it is an image
        self.conv1 = conv(3, conv_dim, 4, batch_norm=False) 
        # 16x16xconv_dim out
        self.conv2 = conv(conv_dim, conv_dim*2, 4)
        # 8x8xconv_dim*2 out
        self.conv3 = conv(conv_dim*2, conv_dim*4, 4)
        # 4x4xconv_dim*4 out
        # final fully connected layer with one output
        self.fc = nn.Linear(4*4*conv_dim*4, 1)
        
        self.dropout = nn.Dropout(0.2)
        

    def forward(self, x):
        """
        Forward propagation of the neural network
        :param x: The input to the neural network     
        :return: Discriminator logits; the output of the neural network
        """
        # define feedforward behavior
        # leaky Relu
        leaky_slope = 0.2
        x = F.leaky_relu(self.conv1(x), leaky_slope)
        x = F.leaky_relu(self.conv2(x), leaky_slope)
        x = F.leaky_relu(self.conv3(x), leaky_slope)
        
        x = self.dropout(x)
        
        # flatten the output into a vector to be fed into fully connected layer
        x = x.view(-1, 4*4*self.conv_dim*4)
        # final layer, no Sigmoid as it is in the loss function nn.BCEWithLogitsLoss()
        x = F.leaky_relu(self.fc(x))
                
        return x


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_discriminator(Discriminator)

Tests Passed

Generator

The generator should upsample an input and generate a new image of the same size as our training data 32x32x3. This should be mostly transpose convolutional layers with normalization applied to the outputs.

Exercise: Complete the Generator class

• The inputs to the generator are vectors of some length z_size
• The output should be a image of shape 32x32x3

# helper function creating a transpose convolutional layer
def deconv(n_inputs, n_outputs, kernel_size, stride=2, padding=1, batch_norm=True):
    """Create convolutional layer with optional batch normalization."""
    layers = []
    transpose_conv_layer = nn.ConvTranspose2d(n_inputs, n_outputs, kernel_size, stride, padding, bias=False)
    
    layers.append(transpose_conv_layer)
    
    if batch_norm:
        # add batch normalization layer
        layers.append(nn.BatchNorm2d(n_outputs))
    
    return nn.Sequential(*layers)

class Generator(nn.Module):
    
    def __init__(self, z_size, conv_dim):
        """
        Initialize the Generator Module
        :param z_size: The length of the input latent vector, z
        :param conv_dim: The depth of the inputs to the *last* transpose convolutional layer
        """
        super(Generator, self).__init__()

        # complete init function
        self.conv_dim = conv_dim # image dimension as generator output
        
        # input is a latent vector z_size
        self.fc = nn.Linear(z_size, 4*4*conv_dim*4)
        
        # transpose convolutional layers
        self.trnsp_conv1 = deconv(conv_dim*4, conv_dim*2, 4)
        self.trnsp_conv2 = deconv(conv_dim*2, conv_dim, 4)
        # no batchnormalization as output is an RGB image
        self.trnsp_conv3 = deconv(conv_dim, 3, 4, batch_norm=False)
        
        self.dropout = nn.Dropout(0.2)
        
    def forward(self, x):
        """
        Forward propagation of the neural network
        :param x: The input to the neural network     
        :return: A 32x32x3 Tensor image as output
        """
        # define feedforward behavior
        x = F.relu(self.fc(x))
        # reshape into (batch_size, 4, 4)
        x = x.view(-1, self.conv_dim*4, 4, 4)
        # transpose convolutional layers
        x = F.relu(self.trnsp_conv1(x))
        x = self.dropout(x)
        x = F.relu(self.trnsp_conv2(x))
        x = self.dropout(x)
        # convert into -1 to 1 image
        x = F.tanh(self.trnsp_conv3(x))
                               
        return x

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_generator(Generator)

Tests Passed

C:\Users\dmitr\anaconda3\lib\site-packages\torch\nn\functional.py:1698: UserWarning: nn.functional.tanh is deprecated. Use torch.tanh instead.
  warnings.warn("nn.functional.tanh is deprecated. Use torch.tanh instead.")

Initialize the weights of your networks

To help your models converge, you should initialize the weights of the convolutional and linear layers in your model. From reading the original DCGAN paper, they say:

All weights were initialized from a zero-centered Normal distribution with standard deviation 0.02.

So, your next task will be to define a weight initialization function that does just this!

You can refer back to the lesson on weight initialization or even consult existing model code, such as that from the networks.py file in CycleGAN Github repository to help you complete this function.

Exercise: Complete the weight initialization function

• This should initialize only convolutional and linear layers
• Initialize the weights to a normal distribution, centered around 0, with a standard deviation of 0.02.
• The bias terms, if they exist, may be left alone or set to 0.

def weights_init_normal(m):
    """
    Applies initial weights to certain layers in a model .
    The weights are taken from a normal distribution 
    with mean = 0, std dev = 0.02.
    :param m: A module or layer in a network    
    """
    # classname will be something like:
    # `Conv`, `BatchNorm2d`, `Linear`, etc.
    mean = 0
    std_dev = 0.02
    classname = m.__class__.__name__
    if classname.find('Conv') != -1 or classname.find('Linear') != -1:
        nn.init.normal_(m.weight.data, mean, std_dev)
    elif classname.find('BatchNorm2d') != -1:
        nn.init.normal_(m.weight.data, 1.0, 0.02)
        nn.init.constant_(m.bias.data, 0)
    
    # TODO: Apply initial weights to convolutional and linear layers
        

Build complete network

Define your models' hyperparameters and instantiate the discriminator and generator from the classes defined above. Make sure you've passed in the correct input arguments.

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
def build_network(d_conv_dim, g_conv_dim, z_size):
    # define discriminator and generator
    D = Discriminator(d_conv_dim)
    G = Generator(z_size=z_size, conv_dim=g_conv_dim)

    # initialize model weights
    D.apply(weights_init_normal)
    G.apply(weights_init_normal)

    print(D)
    print()
    print(G)
    
    return D, G

Exercise: Define model hyperparameters

# Define model hyperparams
d_conv_dim = 128
g_conv_dim = 128
z_size = 128

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
D, G = build_network(d_conv_dim, g_conv_dim, z_size)

Discriminator(
  (conv1): Sequential(
    (0): Conv2d(3, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
  )
  (conv2): Sequential(
    (0): Conv2d(128, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
    (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  )
  (conv3): Sequential(
    (0): Conv2d(256, 512, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
    (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  )
  (fc): Linear(in_features=8192, out_features=1, bias=True)
  (dropout): Dropout(p=0.2, inplace=False)
)

Generator(
  (fc): Linear(in_features=128, out_features=8192, bias=True)
  (trnsp_conv1): Sequential(
    (0): ConvTranspose2d(512, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
    (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  )
  (trnsp_conv2): Sequential(
    (0): ConvTranspose2d(256, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
    (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  )
  (trnsp_conv3): Sequential(
    (0): ConvTranspose2d(128, 3, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
  )
  (dropout): Dropout(p=0.2, inplace=False)
)

Training on GPU

Check if you can train on GPU. Here, we'll set this as a boolean variable train_on_gpu. Later, you'll be responsible for making sure that

• Models,
• Model inputs, and
• Loss function arguments

Are moved to GPU, where appropriate.

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import torch

# Check for a GPU
train_on_gpu = torch.cuda.is_available()
if not train_on_gpu:
    print('No GPU found. Please use a GPU to train your neural network.')
else:
    print('Training on GPU!')

Training on GPU!

---

Discriminator and Generator Losses

Now we need to calculate the losses for both types of adversarial networks.

Discriminator Losses

• For the discriminator, the total loss is the sum of the losses for real and fake images, d_loss = d_real_loss + d_fake_loss.
• Remember that we want the discriminator to output 1 for real images and 0 for fake images, so we need to set up the losses to reflect that.

Generator Loss

The generator loss will look similar only with flipped labels. The generator's goal is to get the discriminator to think its generated images are real.

Exercise: Complete real and fake loss functions

You may choose to use either cross entropy or a least squares error loss to complete the following real_loss and fake_loss functions.

def real_loss(D_out):
    '''Calculates how close discriminator outputs are to being real.
       param, D_out: discriminator logits
       return: real loss'''
    batch_size = D_out.size(0)
    labels = torch.ones(batch_size)*0.9 # use smoothing
    if train_on_gpu:
        labels = labels.cuda()
    criterion = nn.BCEWithLogitsLoss()
    loss = criterion(D_out.squeeze(), labels)
    return loss

def fake_loss(D_out):
    '''Calculates how close discriminator outputs are to being fake.
       param, D_out: discriminator logits
       return: fake loss'''
    batch_size = D_out.size(0)
    labels = torch.zeros(batch_size)
    if train_on_gpu:
        labels = labels.cuda()
    criterion = nn.BCEWithLogitsLoss()
    loss = criterion(D_out.squeeze(), labels)
    return loss

Optimizers

Exercise: Define optimizers for your Discriminator (D) and Generator (G)

Define optimizers for your models with appropriate hyperparameters.

import torch.optim as optim

learning_rate = 0.0002
beta1 = 0.5
beta2 = 0.999

# Create optimizers for the discriminator D and generator G
d_optimizer = optim.Adam(D.parameters(), learning_rate, [beta1, beta2])
g_optimizer = optim.Adam(G.parameters(), learning_rate, [beta1, beta2])

---

Training

Training will involve alternating between training the discriminator and the generator. You'll use your functions real_loss and fake_loss to help you calculate the discriminator losses.

• You should train the discriminator by alternating on real and fake images
• Then the generator, which tries to trick the discriminator and should have an opposing loss function

Saving Samples

You've been given some code to print out some loss statistics and save some generated "fake" samples.

Exercise: Complete the training function

Keep in mind that, if you've moved your models to GPU, you'll also have to move any model inputs to GPU.

def train(D, G, n_epochs, print_every=50):
    '''Trains adversarial networks for some number of epochs
       param, D: the discriminator network
       param, G: the generator network
       param, n_epochs: number of epochs to train for
       param, print_every: when to print and record the models' losses
       return: D and G losses'''
    
    # move models to GPU
    if train_on_gpu:
        D.cuda()
        G.cuda()

    # keep track of loss and generated, "fake" samples
    samples = []
    losses = []

    # Get some fixed data for sampling. These are images that are held
    # constant throughout training, and allow us to inspect the model's performance
    sample_size=16
    fixed_z = np.random.uniform(-1, 1, size=(sample_size, z_size))
    fixed_z = torch.from_numpy(fixed_z).float()
    # move z to GPU if available
    if train_on_gpu:
        fixed_z = fixed_z.cuda()

    # epoch training loop
    for epoch in range(n_epochs):

        # batch training loop
        for batch_i, (real_images, _) in enumerate(celeba_train_loader):

            batch_size = real_images.size(0)
            real_images = scale(real_images)

            # ===============================================
            #         YOUR CODE HERE: TRAIN THE NETWORKS
            # ===============================================
            
            # 1. Train the discriminator on real and fake images
            d_optimizer.zero_grad()
            # do real images
            if train_on_gpu:
                real_images = real_images.cuda()
            D_real = D(real_images)
            d_real_loss = real_loss(D_real)
            
            # do fake images
            z = np.random.uniform(-1, 1, size=(batch_size, z_size))
            z = torch.from_numpy(z).float()
            
            if train_on_gpu:
                z = z.cuda()
            fake_images = G(z)
            D_fake = D(fake_images)
            d_fake_loss = fake_loss(D_fake)
            
            d_loss = d_real_loss + d_fake_loss
            d_loss.backward()
            d_optimizer.step()

            # 2. Train the generator with an adversarial loss
            g_optimizer.zero_grad()
            
            z = np.random.uniform(-1, 1, size=(batch_size, z_size))
            z = torch.from_numpy(z).float()
            
            if train_on_gpu:
                z = z.cuda()
            fake_images = G(z)
            D_fake = D(fake_images)
            g_loss = real_loss(D_fake)
            g_loss.backward()
            g_optimizer.step()
                        
            # ===============================================
            #              END OF YOUR CODE
            # ===============================================

            # Print some loss stats
            if batch_i % print_every == 0:
                # append discriminator loss and generator loss
                losses.append((d_loss.item(), g_loss.item()))
                # print discriminator and generator loss
                print('Epoch [{:5d}/{:5d}] | d_loss: {:6.4f} | g_loss: {:6.4f}'.format(
                        epoch+1, n_epochs, d_loss.item(), g_loss.item()))


        ## AFTER EACH EPOCH##    
        # this code assumes your generator is named G, feel free to change the name
        # generate and save sample, fake images
        G.eval() # for generating samples
        samples_z = G(fixed_z)
        samples.append(samples_z)
        G.train() # back to training mode

    # Save training generator samples
    with open('train_samples.pkl', 'wb') as f:
        pkl.dump(samples, f)
    
    # finally return losses
    return losses

Set your number of training epochs and train your GAN!

from time import time
# set number of epochs 
n_epochs = 10


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# call training function
time_start = time()
losses = train(D, G, n_epochs=n_epochs)
print(f"Training time: {time() - time_start:.1f} seconds")

C:\Users\dmitr\anaconda3\lib\site-packages\torch\nn\functional.py:1698: UserWarning: nn.functional.tanh is deprecated. Use torch.tanh instead.
  warnings.warn("nn.functional.tanh is deprecated. Use torch.tanh instead.")

Epoch [    1/   10] | d_loss: 1.3318 | g_loss: 0.8145
Epoch [    1/   10] | d_loss: 1.3355 | g_loss: 0.8402
Epoch [    1/   10] | d_loss: 1.3243 | g_loss: 0.8189
Epoch [    1/   10] | d_loss: 1.3474 | g_loss: 0.8518
Epoch [    1/   10] | d_loss: 1.3383 | g_loss: 0.8184
Epoch [    1/   10] | d_loss: 1.3418 | g_loss: 0.9352
Epoch [    1/   10] | d_loss: 1.3552 | g_loss: 0.8262
Epoch [    1/   10] | d_loss: 1.3051 | g_loss: 0.8524
Epoch [    1/   10] | d_loss: 1.5071 | g_loss: 0.8586
Epoch [    1/   10] | d_loss: 1.3400 | g_loss: 0.8140
Epoch [    1/   10] | d_loss: 1.3751 | g_loss: 0.8755
Epoch [    1/   10] | d_loss: 1.3631 | g_loss: 0.8178
Epoch [    1/   10] | d_loss: 1.3514 | g_loss: 0.8284
Epoch [    1/   10] | d_loss: 1.3501 | g_loss: 0.8461
Epoch [    1/   10] | d_loss: 1.3236 | g_loss: 0.8430
Epoch [    1/   10] | d_loss: 1.3698 | g_loss: 0.7907
Epoch [    1/   10] | d_loss: 1.3688 | g_loss: 0.8074
Epoch [    1/   10] | d_loss: 1.3588 | g_loss: 0.8068
Epoch [    1/   10] | d_loss: 1.3446 | g_loss: 0.8118
Epoch [    1/   10] | d_loss: 1.3420 | g_loss: 0.8545
Epoch [    1/   10] | d_loss: 1.3159 | g_loss: 0.8467
Epoch [    1/   10] | d_loss: 1.3104 | g_loss: 0.8145
Epoch [    1/   10] | d_loss: 1.3534 | g_loss: 0.8191
Epoch [    1/   10] | d_loss: 1.3555 | g_loss: 0.8160
Epoch [    1/   10] | d_loss: 1.3189 | g_loss: 0.8483
Epoch [    1/   10] | d_loss: 1.3759 | g_loss: 0.8377
Epoch [    1/   10] | d_loss: 1.3622 | g_loss: 0.8029
Epoch [    1/   10] | d_loss: 1.3584 | g_loss: 0.8176
Epoch [    1/   10] | d_loss: 1.3371 | g_loss: 0.8567
Epoch [    1/   10] | d_loss: 1.3821 | g_loss: 0.8028
Epoch [    1/   10] | d_loss: 1.4066 | g_loss: 0.8209
Epoch [    1/   10] | d_loss: 1.3779 | g_loss: 0.7919
Epoch [    1/   10] | d_loss: 1.3702 | g_loss: 0.8294
Epoch [    1/   10] | d_loss: 1.3328 | g_loss: 0.8400
Epoch [    1/   10] | d_loss: 1.3634 | g_loss: 0.8130
Epoch [    1/   10] | d_loss: 1.3731 | g_loss: 0.8309
Epoch [    1/   10] | d_loss: 1.3583 | g_loss: 0.8183
Epoch [    1/   10] | d_loss: 1.3642 | g_loss: 0.8680
Epoch [    1/   10] | d_loss: 1.3634 | g_loss: 0.8053
Epoch [    1/   10] | d_loss: 1.3567 | g_loss: 0.8037
Epoch [    1/   10] | d_loss: 1.3876 | g_loss: 0.8544
Epoch [    1/   10] | d_loss: 1.3605 | g_loss: 0.8495
Epoch [    1/   10] | d_loss: 1.3050 | g_loss: 0.8680
Epoch [    1/   10] | d_loss: 1.3270 | g_loss: 0.8216
Epoch [    1/   10] | d_loss: 1.3342 | g_loss: 0.8390
Epoch [    1/   10] | d_loss: 1.3293 | g_loss: 0.8508
Epoch [    1/   10] | d_loss: 1.3742 | g_loss: 0.7284
Epoch [    1/   10] | d_loss: 1.3602 | g_loss: 0.9399
Epoch [    1/   10] | d_loss: 1.3585 | g_loss: 0.8384
Epoch [    1/   10] | d_loss: 1.3452 | g_loss: 0.8486
Epoch [    1/   10] | d_loss: 1.3204 | g_loss: 0.8437
Epoch [    1/   10] | d_loss: 1.3397 | g_loss: 0.8306
Epoch [    1/   10] | d_loss: 1.3531 | g_loss: 0.8317
Epoch [    1/   10] | d_loss: 1.3403 | g_loss: 0.8117
Epoch [    1/   10] | d_loss: 1.3853 | g_loss: 0.8291
Epoch [    1/   10] | d_loss: 1.3850 | g_loss: 0.8391
Epoch [    1/   10] | d_loss: 1.3187 | g_loss: 0.8745
Epoch [    2/   10] | d_loss: 1.3439 | g_loss: 0.8310
Epoch [    2/   10] | d_loss: 1.3834 | g_loss: 0.7990
Epoch [    2/   10] | d_loss: 1.3636 | g_loss: 0.8153
Epoch [    2/   10] | d_loss: 1.3295 | g_loss: 0.8529
Epoch [    2/   10] | d_loss: 1.2666 | g_loss: 0.8280
Epoch [    2/   10] | d_loss: 1.3235 | g_loss: 0.8445
Epoch [    2/   10] | d_loss: 1.3271 | g_loss: 0.8516
Epoch [    2/   10] | d_loss: 1.2959 | g_loss: 0.8185
Epoch [    2/   10] | d_loss: 1.3026 | g_loss: 0.7852
Epoch [    2/   10] | d_loss: 1.3432 | g_loss: 0.8407
Epoch [    2/   10] | d_loss: 1.3620 | g_loss: 0.8136
Epoch [    2/   10] | d_loss: 1.3622 | g_loss: 0.8616
Epoch [    2/   10] | d_loss: 1.3596 | g_loss: 0.8529
Epoch [    2/   10] | d_loss: 1.3778 | g_loss: 0.8284
Epoch [    2/   10] | d_loss: 1.3854 | g_loss: 0.7820
Epoch [    2/   10] | d_loss: 1.3684 | g_loss: 0.7866
Epoch [    2/   10] | d_loss: 1.3736 | g_loss: 0.7902
Epoch [    2/   10] | d_loss: 1.3639 | g_loss: 0.8080
Epoch [    2/   10] | d_loss: 1.3761 | g_loss: 0.7995
Epoch [    2/   10] | d_loss: 1.3731 | g_loss: 0.8176
Epoch [    2/   10] | d_loss: 1.3580 | g_loss: 0.8243
Epoch [    2/   10] | d_loss: 1.3440 | g_loss: 0.8757
Epoch [    2/   10] | d_loss: 1.3026 | g_loss: 0.8382
Epoch [    2/   10] | d_loss: 1.3804 | g_loss: 0.8523
Epoch [    2/   10] | d_loss: 1.3131 | g_loss: 0.8563
Epoch [    2/   10] | d_loss: 1.3970 | g_loss: 0.8095
Epoch [    2/   10] | d_loss: 1.3602 | g_loss: 0.8091
Epoch [    2/   10] | d_loss: 1.3646 | g_loss: 0.7958
Epoch [    2/   10] | d_loss: 1.3544 | g_loss: 0.8627
Epoch [    2/   10] | d_loss: 1.3295 | g_loss: 0.8566
Epoch [    2/   10] | d_loss: 1.3770 | g_loss: 0.8404
Epoch [    2/   10] | d_loss: 1.3641 | g_loss: 0.8442
Epoch [    2/   10] | d_loss: 1.3745 | g_loss: 0.8324
Epoch [    2/   10] | d_loss: 1.3549 | g_loss: 0.8390
Epoch [    2/   10] | d_loss: 1.3432 | g_loss: 0.8182
Epoch [    2/   10] | d_loss: 1.3627 | g_loss: 0.8027
Epoch [    2/   10] | d_loss: 1.3353 | g_loss: 0.8516
Epoch [    2/   10] | d_loss: 1.3228 | g_loss: 0.8596
Epoch [    2/   10] | d_loss: 1.3538 | g_loss: 0.9008
Epoch [    2/   10] | d_loss: 1.3910 | g_loss: 0.8004
Epoch [    2/   10] | d_loss: 1.4128 | g_loss: 0.7817
Epoch [    2/   10] | d_loss: 1.3770 | g_loss: 0.7968
Epoch [    2/   10] | d_loss: 1.3706 | g_loss: 0.8225
Epoch [    2/   10] | d_loss: 1.3944 | g_loss: 0.8187
Epoch [    2/   10] | d_loss: 1.3907 | g_loss: 0.8072
Epoch [    2/   10] | d_loss: 1.3489 | g_loss: 0.7978
Epoch [    2/   10] | d_loss: 1.3720 | g_loss: 0.8142
Epoch [    2/   10] | d_loss: 1.3783 | g_loss: 0.8102
Epoch [    2/   10] | d_loss: 1.3452 | g_loss: 0.8160
Epoch [    2/   10] | d_loss: 1.3547 | g_loss: 0.8131
Epoch [    2/   10] | d_loss: 1.3549 | g_loss: 0.8540
Epoch [    2/   10] | d_loss: 1.4011 | g_loss: 0.8675
Epoch [    2/   10] | d_loss: 1.3222 | g_loss: 0.8515
Epoch [    2/   10] | d_loss: 1.3459 | g_loss: 0.8420
Epoch [    2/   10] | d_loss: 1.3401 | g_loss: 0.8297
Epoch [    2/   10] | d_loss: 1.3969 | g_loss: 0.8568
Epoch [    2/   10] | d_loss: 1.3483 | g_loss: 0.7919
Epoch [    3/   10] | d_loss: 1.3768 | g_loss: 0.7982
Epoch [    3/   10] | d_loss: 1.3374 | g_loss: 0.8628
Epoch [    3/   10] | d_loss: 1.3333 | g_loss: 0.8641
Epoch [    3/   10] | d_loss: 1.3261 | g_loss: 0.8072
Epoch [    3/   10] | d_loss: 1.3660 | g_loss: 0.8610
Epoch [    3/   10] | d_loss: 1.3306 | g_loss: 0.8528
Epoch [    3/   10] | d_loss: 1.3741 | g_loss: 0.8580
Epoch [    3/   10] | d_loss: 1.3917 | g_loss: 0.8045
Epoch [    3/   10] | d_loss: 1.3868 | g_loss: 0.7949
Epoch [    3/   10] | d_loss: 1.3974 | g_loss: 0.7990
Epoch [    3/   10] | d_loss: 1.3765 | g_loss: 0.7923
Epoch [    3/   10] | d_loss: 1.3724 | g_loss: 0.7811
Epoch [    3/   10] | d_loss: 1.3772 | g_loss: 0.7883
Epoch [    3/   10] | d_loss: 1.3698 | g_loss: 0.8118
Epoch [    3/   10] | d_loss: 1.3733 | g_loss: 0.8211
Epoch [    3/   10] | d_loss: 1.3606 | g_loss: 0.8100
Epoch [    3/   10] | d_loss: 1.3657 | g_loss: 0.8140
Epoch [    3/   10] | d_loss: 1.3454 | g_loss: 0.8176
Epoch [    3/   10] | d_loss: 1.3518 | g_loss: 0.8134
Epoch [    3/   10] | d_loss: 1.3523 | g_loss: 0.8191
Epoch [    3/   10] | d_loss: 1.3860 | g_loss: 0.9303
Epoch [    3/   10] | d_loss: 1.3626 | g_loss: 0.8511
Epoch [    3/   10] | d_loss: 1.3521 | g_loss: 0.8565
Epoch [    3/   10] | d_loss: 1.3665 | g_loss: 0.8128
Epoch [    3/   10] | d_loss: 1.3007 | g_loss: 0.8294
Epoch [    3/   10] | d_loss: 1.3590 | g_loss: 0.7876
Epoch [    3/   10] | d_loss: 1.3787 | g_loss: 0.7997
Epoch [    3/   10] | d_loss: 1.3598 | g_loss: 0.8431
Epoch [    3/   10] | d_loss: 1.3311 | g_loss: 0.8690
Epoch [    3/   10] | d_loss: 1.3436 | g_loss: 0.8412
Epoch [    3/   10] | d_loss: 1.3512 | g_loss: 0.8574
Epoch [    3/   10] | d_loss: 1.3437 | g_loss: 0.8344
Epoch [    3/   10] | d_loss: 1.3886 | g_loss: 0.8251
Epoch [    3/   10] | d_loss: 1.4862 | g_loss: 0.8410
Epoch [    3/   10] | d_loss: 1.3832 | g_loss: 0.8198
Epoch [    3/   10] | d_loss: 1.3618 | g_loss: 0.8751
Epoch [    3/   10] | d_loss: 1.3473 | g_loss: 0.8370
Epoch [    3/   10] | d_loss: 1.3537 | g_loss: 0.8271
Epoch [    3/   10] | d_loss: 1.3121 | g_loss: 0.8459
Epoch [    3/   10] | d_loss: 1.3972 | g_loss: 0.8590
Epoch [    3/   10] | d_loss: 1.3650 | g_loss: 0.8100
Epoch [    3/   10] | d_loss: 1.3787 | g_loss: 0.8736
Epoch [    3/   10] | d_loss: 1.3771 | g_loss: 0.7962
Epoch [    3/   10] | d_loss: 1.3772 | g_loss: 0.8099
Epoch [    3/   10] | d_loss: 1.3758 | g_loss: 0.8061
Epoch [    3/   10] | d_loss: 1.3793 | g_loss: 0.7890
Epoch [    3/   10] | d_loss: 1.3604 | g_loss: 0.8351
Epoch [    3/   10] | d_loss: 1.3655 | g_loss: 0.7896
Epoch [    3/   10] | d_loss: 1.3845 | g_loss: 0.8602
Epoch [    3/   10] | d_loss: 1.3729 | g_loss: 0.8090
Epoch [    3/   10] | d_loss: 1.3832 | g_loss: 0.8634
Epoch [    3/   10] | d_loss: 1.3425 | g_loss: 0.8220
Epoch [    3/   10] | d_loss: 1.3665 | g_loss: 0.8376
Epoch [    3/   10] | d_loss: 1.3184 | g_loss: 0.8173
Epoch [    3/   10] | d_loss: 1.3277 | g_loss: 0.8412
Epoch [    3/   10] | d_loss: 1.3817 | g_loss: 0.8928
Epoch [    3/   10] | d_loss: 1.4081 | g_loss: 0.8699
Epoch [    4/   10] | d_loss: 1.3366 | g_loss: 0.8630
Epoch [    4/   10] | d_loss: 1.3471 | g_loss: 0.8575
Epoch [    4/   10] | d_loss: 1.3551 | g_loss: 0.8190
Epoch [    4/   10] | d_loss: 1.3201 | g_loss: 0.8088
Epoch [    4/   10] | d_loss: 1.3909 | g_loss: 0.7886
Epoch [    4/   10] | d_loss: 1.4055 | g_loss: 0.8488
Epoch [    4/   10] | d_loss: 1.3741 | g_loss: 0.8114
Epoch [    4/   10] | d_loss: 1.3762 | g_loss: 0.7995
Epoch [    4/   10] | d_loss: 1.3555 | g_loss: 0.8108
Epoch [    4/   10] | d_loss: 1.3608 | g_loss: 0.8185
Epoch [    4/   10] | d_loss: 1.3731 | g_loss: 0.8353
Epoch [    4/   10] | d_loss: 1.3204 | g_loss: 0.8722
Epoch [    4/   10] | d_loss: 1.3322 | g_loss: 0.8391
Epoch [    4/   10] | d_loss: 1.3272 | g_loss: 0.8482
Epoch [    4/   10] | d_loss: 1.2970 | g_loss: 0.8512
Epoch [    4/   10] | d_loss: 1.3377 | g_loss: 0.8543
Epoch [    4/   10] | d_loss: 1.3731 | g_loss: 0.8637
Epoch [    4/   10] | d_loss: 1.3528 | g_loss: 0.8181
Epoch [    4/   10] | d_loss: 1.3307 | g_loss: 0.8610
Epoch [    4/   10] | d_loss: 1.3533 | g_loss: 0.8256
Epoch [    4/   10] | d_loss: 1.3807 | g_loss: 0.8327
Epoch [    4/   10] | d_loss: 1.3126 | g_loss: 0.8450
Epoch [    4/   10] | d_loss: 1.3720 | g_loss: 0.8328
Epoch [    4/   10] | d_loss: 1.3748 | g_loss: 0.8208
Epoch [    4/   10] | d_loss: 1.3987 | g_loss: 0.8507
Epoch [    4/   10] | d_loss: 1.3890 | g_loss: 0.8174
Epoch [    4/   10] | d_loss: 1.3840 | g_loss: 0.7941
Epoch [    4/   10] | d_loss: 1.3839 | g_loss: 0.7933
Epoch [    4/   10] | d_loss: 1.3884 | g_loss: 0.8125
Epoch [    4/   10] | d_loss: 1.3779 | g_loss: 0.8083
Epoch [    4/   10] | d_loss: 1.3902 | g_loss: 0.8088
Epoch [    4/   10] | d_loss: 1.3806 | g_loss: 0.8088
Epoch [    4/   10] | d_loss: 1.3947 | g_loss: 0.7885
Epoch [    4/   10] | d_loss: 1.3904 | g_loss: 0.7900
Epoch [    4/   10] | d_loss: 1.3754 | g_loss: 0.7872
Epoch [    4/   10] | d_loss: 1.3773 | g_loss: 0.7924
Epoch [    4/   10] | d_loss: 1.3760 | g_loss: 0.7871
Epoch [    4/   10] | d_loss: 1.3822 | g_loss: 0.8072
Epoch [    4/   10] | d_loss: 1.3615 | g_loss: 0.7966
Epoch [    4/   10] | d_loss: 1.3650 | g_loss: 0.8124
Epoch [    4/   10] | d_loss: 1.3645 | g_loss: 0.8167
Epoch [    4/   10] | d_loss: 1.3304 | g_loss: 0.8214
Epoch [    4/   10] | d_loss: 1.3717 | g_loss: 0.8272
Epoch [    4/   10] | d_loss: 1.3950 | g_loss: 0.8232
Epoch [    4/   10] | d_loss: 1.3544 | g_loss: 0.8155
Epoch [    4/   10] | d_loss: 1.3829 | g_loss: 0.8308
Epoch [    4/   10] | d_loss: 1.3830 | g_loss: 0.8051
Epoch [    4/   10] | d_loss: 1.3918 | g_loss: 0.8168
Epoch [    4/   10] | d_loss: 1.3799 | g_loss: 0.8028
Epoch [    4/   10] | d_loss: 1.3885 | g_loss: 0.8036
Epoch [    4/   10] | d_loss: 1.3814 | g_loss: 0.8191
Epoch [    4/   10] | d_loss: 1.3921 | g_loss: 0.8418
Epoch [    4/   10] | d_loss: 1.3688 | g_loss: 0.8075
Epoch [    4/   10] | d_loss: 1.3658 | g_loss: 0.8199
Epoch [    4/   10] | d_loss: 1.3589 | g_loss: 0.8396
Epoch [    4/   10] | d_loss: 1.3395 | g_loss: 0.8527
Epoch [    4/   10] | d_loss: 1.3495 | g_loss: 0.8621
Epoch [    5/   10] | d_loss: 1.3194 | g_loss: 0.8407
Epoch [    5/   10] | d_loss: 1.3323 | g_loss: 0.8349
Epoch [    5/   10] | d_loss: 1.3059 | g_loss: 0.8412
Epoch [    5/   10] | d_loss: 1.3811 | g_loss: 0.8223
Epoch [    5/   10] | d_loss: 1.3561 | g_loss: 0.7951
Epoch [    5/   10] | d_loss: 1.3948 | g_loss: 0.8109
Epoch [    5/   10] | d_loss: 1.3604 | g_loss: 0.8273
Epoch [    5/   10] | d_loss: 1.3552 | g_loss: 0.8470
Epoch [    5/   10] | d_loss: 1.3545 | g_loss: 0.8211
Epoch [    5/   10] | d_loss: 1.3583 | g_loss: 0.8491
Epoch [    5/   10] | d_loss: 1.3464 | g_loss: 0.8500
Epoch [    5/   10] | d_loss: 1.3916 | g_loss: 0.8102
Epoch [    5/   10] | d_loss: 1.3549 | g_loss: 0.8290
Epoch [    5/   10] | d_loss: 1.3281 | g_loss: 0.8459
Epoch [    5/   10] | d_loss: 1.3585 | g_loss: 0.8175
Epoch [    5/   10] | d_loss: 1.3616 | g_loss: 0.8396
Epoch [    5/   10] | d_loss: 1.3609 | g_loss: 0.8297
Epoch [    5/   10] | d_loss: 1.2996 | g_loss: 0.8154
Epoch [    5/   10] | d_loss: 1.3927 | g_loss: 0.8214
Epoch [    5/   10] | d_loss: 1.3890 | g_loss: 0.7873
Epoch [    5/   10] | d_loss: 1.3746 | g_loss: 0.7893
Epoch [    5/   10] | d_loss: 1.3792 | g_loss: 0.8011
Epoch [    5/   10] | d_loss: 1.3755 | g_loss: 0.7784
Epoch [    5/   10] | d_loss: 1.3737 | g_loss: 0.7964
Epoch [    5/   10] | d_loss: 1.3776 | g_loss: 0.7865
Epoch [    5/   10] | d_loss: 1.3778 | g_loss: 0.7846
Epoch [    5/   10] | d_loss: 1.3793 | g_loss: 0.7804
Epoch [    5/   10] | d_loss: 1.3802 | g_loss: 0.7873
Epoch [    5/   10] | d_loss: 1.3739 | g_loss: 0.7906
Epoch [    5/   10] | d_loss: 1.3770 | g_loss: 0.8053
Epoch [    5/   10] | d_loss: 1.3828 | g_loss: 0.7917
Epoch [    5/   10] | d_loss: 1.3597 | g_loss: 0.7869
Epoch [    5/   10] | d_loss: 1.3750 | g_loss: 0.7949
Epoch [    5/   10] | d_loss: 1.3698 | g_loss: 0.7964
Epoch [    5/   10] | d_loss: 1.3758 | g_loss: 0.7889
Epoch [    5/   10] | d_loss: 1.3671 | g_loss: 0.7870
Epoch [    5/   10] | d_loss: 1.3729 | g_loss: 0.8010
Epoch [    5/   10] | d_loss: 1.3724 | g_loss: 0.8061
Epoch [    5/   10] | d_loss: 1.3744 | g_loss: 0.8232
Epoch [    5/   10] | d_loss: 1.3631 | g_loss: 0.7888
Epoch [    5/   10] | d_loss: 1.3474 | g_loss: 0.8108
Epoch [    5/   10] | d_loss: 1.3588 | g_loss: 0.8539
Epoch [    5/   10] | d_loss: 1.3516 | g_loss: 0.8759
Epoch [    5/   10] | d_loss: 1.3533 | g_loss: 0.8655
Epoch [    5/   10] | d_loss: 1.3830 | g_loss: 0.8620
Epoch [    5/   10] | d_loss: 1.3751 | g_loss: 0.8120
Epoch [    5/   10] | d_loss: 1.3562 | g_loss: 0.8080
Epoch [    5/   10] | d_loss: 1.3727 | g_loss: 0.7941
Epoch [    5/   10] | d_loss: 1.3883 | g_loss: 0.7888
Epoch [    5/   10] | d_loss: 1.3827 | g_loss: 0.8226
Epoch [    5/   10] | d_loss: 1.3804 | g_loss: 0.8251
Epoch [    5/   10] | d_loss: 1.3453 | g_loss: 0.8216
Epoch [    5/   10] | d_loss: 1.3886 | g_loss: 0.8650
Epoch [    5/   10] | d_loss: 1.3514 | g_loss: 0.8169
Epoch [    5/   10] | d_loss: 1.3309 | g_loss: 0.8825
Epoch [    5/   10] | d_loss: 1.3471 | g_loss: 0.8405
Epoch [    5/   10] | d_loss: 1.3447 | g_loss: 0.8280
Epoch [    6/   10] | d_loss: 1.2691 | g_loss: 0.7382
Epoch [    6/   10] | d_loss: 1.4140 | g_loss: 0.8654
Epoch [    6/   10] | d_loss: 1.4036 | g_loss: 0.8256
Epoch [    6/   10] | d_loss: 1.3809 | g_loss: 0.8100
Epoch [    6/   10] | d_loss: 1.3669 | g_loss: 0.8013
Epoch [    6/   10] | d_loss: 1.3756 | g_loss: 0.7950
Epoch [    6/   10] | d_loss: 1.3775 | g_loss: 0.7987
Epoch [    6/   10] | d_loss: 1.3798 | g_loss: 0.8071
Epoch [    6/   10] | d_loss: 1.3771 | g_loss: 0.8027
Epoch [    6/   10] | d_loss: 1.3719 | g_loss: 0.7953
Epoch [    6/   10] | d_loss: 1.3537 | g_loss: 0.8071
Epoch [    6/   10] | d_loss: 1.3635 | g_loss: 0.8323
Epoch [    6/   10] | d_loss: 1.3691 | g_loss: 0.8207
Epoch [    6/   10] | d_loss: 1.3598 | g_loss: 0.8324
Epoch [    6/   10] | d_loss: 1.3515 | g_loss: 0.8488
Epoch [    6/   10] | d_loss: 1.3553 | g_loss: 0.8527
Epoch [    6/   10] | d_loss: 1.3269 | g_loss: 0.8495
Epoch [    6/   10] | d_loss: 1.3397 | g_loss: 0.8656
Epoch [    6/   10] | d_loss: 1.3418 | g_loss: 0.8106
Epoch [    6/   10] | d_loss: 1.3126 | g_loss: 0.8327
Epoch [    6/   10] | d_loss: 1.3356 | g_loss: 0.8473
Epoch [    6/   10] | d_loss: 1.4031 | g_loss: 0.8158
Epoch [    6/   10] | d_loss: 1.3748 | g_loss: 0.7743
Epoch [    6/   10] | d_loss: 1.3791 | g_loss: 0.7830
Epoch [    6/   10] | d_loss: 1.3671 | g_loss: 0.8005
Epoch [    6/   10] | d_loss: 1.3667 | g_loss: 0.8034
Epoch [    6/   10] | d_loss: 1.3760 | g_loss: 0.7972
Epoch [    6/   10] | d_loss: 1.3678 | g_loss: 0.8002
Epoch [    6/   10] | d_loss: 1.3785 | g_loss: 0.7961
Epoch [    6/   10] | d_loss: 1.3859 | g_loss: 0.7923
Epoch [    6/   10] | d_loss: 1.3539 | g_loss: 0.7981
Epoch [    6/   10] | d_loss: 1.3645 | g_loss: 0.8237
Epoch [    6/   10] | d_loss: 1.3648 | g_loss: 0.8034
Epoch [    6/   10] | d_loss: 1.3432 | g_loss: 0.8043
Epoch [    6/   10] | d_loss: 1.3494 | g_loss: 0.8373
Epoch [    6/   10] | d_loss: 1.3407 | g_loss: 0.8120
Epoch [    6/   10] | d_loss: 1.3402 | g_loss: 0.8541
Epoch [    6/   10] | d_loss: 1.3611 | g_loss: 0.8575
Epoch [    6/   10] | d_loss: 1.3500 | g_loss: 0.8493
Epoch [    6/   10] | d_loss: 1.2896 | g_loss: 0.8085
Epoch [    6/   10] | d_loss: 1.3559 | g_loss: 0.8344
Epoch [    6/   10] | d_loss: 1.3523 | g_loss: 0.8480
Epoch [    6/   10] | d_loss: 1.3219 | g_loss: 0.8433
Epoch [    6/   10] | d_loss: 1.3798 | g_loss: 0.9158
Epoch [    6/   10] | d_loss: 1.3802 | g_loss: 0.7888
Epoch [    6/   10] | d_loss: 1.3887 | g_loss: 0.7867
Epoch [    6/   10] | d_loss: 1.3769 | g_loss: 0.8052
Epoch [    6/   10] | d_loss: 1.3709 | g_loss: 0.8470
Epoch [    6/   10] | d_loss: 1.3958 | g_loss: 0.8003
Epoch [    6/   10] | d_loss: 1.3647 | g_loss: 0.8080
Epoch [    6/   10] | d_loss: 1.3638 | g_loss: 0.8133
Epoch [    6/   10] | d_loss: 1.3582 | g_loss: 0.8020
Epoch [    6/   10] | d_loss: 1.3674 | g_loss: 0.8243
Epoch [    6/   10] | d_loss: 1.3714 | g_loss: 0.8319
Epoch [    6/   10] | d_loss: 1.3337 | g_loss: 0.8952
Epoch [    6/   10] | d_loss: 1.2507 | g_loss: 0.7750
Epoch [    6/   10] | d_loss: 1.3697 | g_loss: 0.8128
Epoch [    7/   10] | d_loss: 1.3726 | g_loss: 0.8111
Epoch [    7/   10] | d_loss: 1.3617 | g_loss: 0.7910
Epoch [    7/   10] | d_loss: 1.3578 | g_loss: 0.8248
Epoch [    7/   10] | d_loss: 1.3578 | g_loss: 0.9254
Epoch [    7/   10] | d_loss: 1.3581 | g_loss: 0.8561
Epoch [    7/   10] | d_loss: 1.3405 | g_loss: 0.8296
Epoch [    7/   10] | d_loss: 1.3085 | g_loss: 0.8463
Epoch [    7/   10] | d_loss: 1.3701 | g_loss: 0.8165
Epoch [    7/   10] | d_loss: 1.3789 | g_loss: 0.8301
Epoch [    7/   10] | d_loss: 1.3675 | g_loss: 0.8146
Epoch [    7/   10] | d_loss: 1.3474 | g_loss: 0.8718
Epoch [    7/   10] | d_loss: 1.3583 | g_loss: 0.8444
Epoch [    7/   10] | d_loss: 1.4078 | g_loss: 0.8288
Epoch [    7/   10] | d_loss: 1.3870 | g_loss: 0.7973
Epoch [    7/   10] | d_loss: 1.3851 | g_loss: 0.7968
Epoch [    7/   10] | d_loss: 1.3712 | g_loss: 0.8019
Epoch [    7/   10] | d_loss: 1.3765 | g_loss: 0.8221
Epoch [    7/   10] | d_loss: 1.3293 | g_loss: 0.8171
Epoch [    7/   10] | d_loss: 1.3969 | g_loss: 0.8406
Epoch [    7/   10] | d_loss: 1.3548 | g_loss: 0.8025
Epoch [    7/   10] | d_loss: 1.3454 | g_loss: 0.8380
Epoch [    7/   10] | d_loss: 1.3425 | g_loss: 0.8348
Epoch [    7/   10] | d_loss: 1.3291 | g_loss: 0.8420
Epoch [    7/   10] | d_loss: 1.3423 | g_loss: 0.8239
Epoch [    7/   10] | d_loss: 1.3457 | g_loss: 0.8543
Epoch [    7/   10] | d_loss: 1.3833 | g_loss: 0.8482
Epoch [    7/   10] | d_loss: 1.3510 | g_loss: 0.7973
Epoch [    7/   10] | d_loss: 1.3922 | g_loss: 0.8619
Epoch [    7/   10] | d_loss: 1.3518 | g_loss: 0.8376
Epoch [    7/   10] | d_loss: 1.3076 | g_loss: 0.8619
Epoch [    7/   10] | d_loss: 1.3040 | g_loss: 0.8441
Epoch [    7/   10] | d_loss: 1.3219 | g_loss: 0.8215
Epoch [    7/   10] | d_loss: 1.3494 | g_loss: 0.8055
Epoch [    7/   10] | d_loss: 1.3423 | g_loss: 0.8435
Epoch [    7/   10] | d_loss: 1.3590 | g_loss: 0.8167
Epoch [    7/   10] | d_loss: 1.3572 | g_loss: 0.8338
Epoch [    7/   10] | d_loss: 1.3633 | g_loss: 0.8664
Epoch [    7/   10] | d_loss: 1.3414 | g_loss: 0.8292
Epoch [    7/   10] | d_loss: 1.3249 | g_loss: 0.8556
Epoch [    7/   10] | d_loss: 1.3930 | g_loss: 0.8822
Epoch [    7/   10] | d_loss: 1.3841 | g_loss: 0.7859
Epoch [    7/   10] | d_loss: 1.3865 | g_loss: 0.7863
Epoch [    7/   10] | d_loss: 1.3755 | g_loss: 0.7994
Epoch [    7/   10] | d_loss: 1.3737 | g_loss: 0.8082
Epoch [    7/   10] | d_loss: 1.3758 | g_loss: 0.8040
Epoch [    7/   10] | d_loss: 1.3705 | g_loss: 0.8117
Epoch [    7/   10] | d_loss: 1.3696 | g_loss: 0.8033
Epoch [    7/   10] | d_loss: 1.3774 | g_loss: 0.8006
Epoch [    7/   10] | d_loss: 1.3682 | g_loss: 0.8239
Epoch [    7/   10] | d_loss: 1.3672 | g_loss: 0.8007
Epoch [    7/   10] | d_loss: 1.3582 | g_loss: 0.8389
Epoch [    7/   10] | d_loss: 1.2932 | g_loss: 0.8593
Epoch [    7/   10] | d_loss: 1.3203 | g_loss: 0.8086
Epoch [    7/   10] | d_loss: 1.3218 | g_loss: 0.8448
Epoch [    7/   10] | d_loss: 1.3359 | g_loss: 0.8676
Epoch [    7/   10] | d_loss: 1.3155 | g_loss: 0.9000
Epoch [    7/   10] | d_loss: 1.3545 | g_loss: 0.8599
Epoch [    8/   10] | d_loss: 1.3467 | g_loss: 0.8630
Epoch [    8/   10] | d_loss: 1.2731 | g_loss: 0.7910
Epoch [    8/   10] | d_loss: 1.3795 | g_loss: 0.8040
Epoch [    8/   10] | d_loss: 1.3889 | g_loss: 0.7889
Epoch [    8/   10] | d_loss: 1.3785 | g_loss: 0.8205
Epoch [    8/   10] | d_loss: 1.3875 | g_loss: 0.7807
Epoch [    8/   10] | d_loss: 1.3688 | g_loss: 0.7893
Epoch [    8/   10] | d_loss: 1.3741 | g_loss: 0.8088
Epoch [    8/   10] | d_loss: 1.3644 | g_loss: 0.8549
Epoch [    8/   10] | d_loss: 1.3602 | g_loss: 0.8260
Epoch [    8/   10] | d_loss: 1.3193 | g_loss: 0.8206
Epoch [    8/   10] | d_loss: 1.3367 | g_loss: 0.8560
Epoch [    8/   10] | d_loss: 1.3908 | g_loss: 0.8112
Epoch [    8/   10] | d_loss: 1.3357 | g_loss: 0.8610
Epoch [    8/   10] | d_loss: 1.3467 | g_loss: 0.8210
Epoch [    8/   10] | d_loss: 1.3120 | g_loss: 0.8675
Epoch [    8/   10] | d_loss: 1.3720 | g_loss: 0.8765
Epoch [    8/   10] | d_loss: 1.3860 | g_loss: 0.8872
Epoch [    8/   10] | d_loss: 1.4085 | g_loss: 0.8433
Epoch [    8/   10] | d_loss: 1.3890 | g_loss: 0.8164
Epoch [    8/   10] | d_loss: 1.3460 | g_loss: 0.8504
Epoch [    8/   10] | d_loss: 1.3487 | g_loss: 0.7894
Epoch [    8/   10] | d_loss: 1.3748 | g_loss: 0.7983
Epoch [    8/   10] | d_loss: 1.3700 | g_loss: 0.8647
Epoch [    8/   10] | d_loss: 1.3780 | g_loss: 0.8045
Epoch [    8/   10] | d_loss: 1.3438 | g_loss: 0.8306
Epoch [    8/   10] | d_loss: 1.3748 | g_loss: 0.8405
Epoch [    8/   10] | d_loss: 1.3524 | g_loss: 0.8086
Epoch [    8/   10] | d_loss: 1.3398 | g_loss: 0.8421
Epoch [    8/   10] | d_loss: 1.3413 | g_loss: 0.8779
Epoch [    8/   10] | d_loss: 1.3333 | g_loss: 0.8101
Epoch [    8/   10] | d_loss: 1.3003 | g_loss: 0.8552
Epoch [    8/   10] | d_loss: 1.3216 | g_loss: 0.8580
Epoch [    8/   10] | d_loss: 1.2918 | g_loss: 0.8444
Epoch [    8/   10] | d_loss: 1.3879 | g_loss: 0.8148
Epoch [    8/   10] | d_loss: 1.3606 | g_loss: 0.8449
Epoch [    8/   10] | d_loss: 1.3924 | g_loss: 0.8206
Epoch [    8/   10] | d_loss: 1.3842 | g_loss: 0.7898
Epoch [    8/   10] | d_loss: 1.3788 | g_loss: 0.7960
Epoch [    8/   10] | d_loss: 1.3825 | g_loss: 0.7971
Epoch [    8/   10] | d_loss: 1.3719 | g_loss: 0.8011
Epoch [    8/   10] | d_loss: 1.3729 | g_loss: 0.8101
Epoch [    8/   10] | d_loss: 1.3750 | g_loss: 0.8018
Epoch [    8/   10] | d_loss: 1.3607 | g_loss: 0.7828
Epoch [    8/   10] | d_loss: 1.3568 | g_loss: 0.8048
Epoch [    8/   10] | d_loss: 1.3592 | g_loss: 0.8251
Epoch [    8/   10] | d_loss: 1.3163 | g_loss: 0.8553
Epoch [    8/   10] | d_loss: 1.3320 | g_loss: 0.8460
Epoch [    8/   10] | d_loss: 1.3692 | g_loss: 0.8155
Epoch [    8/   10] | d_loss: 1.3712 | g_loss: 0.8276
Epoch [    8/   10] | d_loss: 1.3502 | g_loss: 0.8567
Epoch [    8/   10] | d_loss: 1.3462 | g_loss: 0.8764
Epoch [    8/   10] | d_loss: 1.3228 | g_loss: 0.8670
Epoch [    8/   10] | d_loss: 1.3443 | g_loss: 0.8638
Epoch [    8/   10] | d_loss: 1.3809 | g_loss: 0.8084
Epoch [    8/   10] | d_loss: 1.3750 | g_loss: 0.8038
Epoch [    8/   10] | d_loss: 1.3528 | g_loss: 0.8632
Epoch [    9/   10] | d_loss: 1.3396 | g_loss: 0.8541
Epoch [    9/   10] | d_loss: 1.3816 | g_loss: 0.8664
Epoch [    9/   10] | d_loss: 1.3364 | g_loss: 0.8426
Epoch [    9/   10] | d_loss: 1.3296 | g_loss: 0.8375
Epoch [    9/   10] | d_loss: 1.3532 | g_loss: 0.8551
Epoch [    9/   10] | d_loss: 1.2924 | g_loss: 0.6955
Epoch [    9/   10] | d_loss: 1.3408 | g_loss: 0.8318
Epoch [    9/   10] | d_loss: 1.3871 | g_loss: 0.8280
Epoch [    9/   10] | d_loss: 1.3361 | g_loss: 0.8564
Epoch [    9/   10] | d_loss: 1.2970 | g_loss: 0.8536
Epoch [    9/   10] | d_loss: 1.3772 | g_loss: 0.8196
Epoch [    9/   10] | d_loss: 1.3753 | g_loss: 0.8398
Epoch [    9/   10] | d_loss: 1.3419 | g_loss: 0.7727
Epoch [    9/   10] | d_loss: 1.3877 | g_loss: 0.8391
Epoch [    9/   10] | d_loss: 1.3617 | g_loss: 0.8185
Epoch [    9/   10] | d_loss: 1.3373 | g_loss: 0.8428
Epoch [    9/   10] | d_loss: 1.3767 | g_loss: 0.8572
Epoch [    9/   10] | d_loss: 1.3811 | g_loss: 0.7978
Epoch [    9/   10] | d_loss: 1.3853 | g_loss: 0.8472
Epoch [    9/   10] | d_loss: 1.3882 | g_loss: 0.7896
Epoch [    9/   10] | d_loss: 1.3725 | g_loss: 0.8132
Epoch [    9/   10] | d_loss: 1.3492 | g_loss: 0.8308
Epoch [    9/   10] | d_loss: 1.3371 | g_loss: 0.8132
Epoch [    9/   10] | d_loss: 1.3464 | g_loss: 0.8346
Epoch [    9/   10] | d_loss: 1.3625 | g_loss: 0.8196
Epoch [    9/   10] | d_loss: 1.3561 | g_loss: 0.8610
Epoch [    9/   10] | d_loss: 1.3716 | g_loss: 0.8280
Epoch [    9/   10] | d_loss: 1.3148 | g_loss: 0.8443
Epoch [    9/   10] | d_loss: 1.3255 | g_loss: 0.8539
Epoch [    9/   10] | d_loss: 1.3224 | g_loss: 0.8753
Epoch [    9/   10] | d_loss: 1.3674 | g_loss: 0.8464
Epoch [    9/   10] | d_loss: 1.3136 | g_loss: 0.9073
Epoch [    9/   10] | d_loss: 1.3510 | g_loss: 0.8304
Epoch [    9/   10] | d_loss: 1.3656 | g_loss: 0.8918
Epoch [    9/   10] | d_loss: 1.3989 | g_loss: 0.8494
Epoch [    9/   10] | d_loss: 1.4021 | g_loss: 0.7996
Epoch [    9/   10] | d_loss: 1.3584 | g_loss: 0.7933
Epoch [    9/   10] | d_loss: 1.3592 | g_loss: 0.8536
Epoch [    9/   10] | d_loss: 1.3242 | g_loss: 0.8689
Epoch [    9/   10] | d_loss: 1.3175 | g_loss: 0.8409
Epoch [    9/   10] | d_loss: 1.3344 | g_loss: 0.8490
Epoch [    9/   10] | d_loss: 1.3353 | g_loss: 0.8300
Epoch [    9/   10] | d_loss: 1.3030 | g_loss: 0.8622
Epoch [    9/   10] | d_loss: 1.3565 | g_loss: 0.8595
Epoch [    9/   10] | d_loss: 1.3189 | g_loss: 0.8291
Epoch [    9/   10] | d_loss: 1.3703 | g_loss: 0.9218
Epoch [    9/   10] | d_loss: 1.3438 | g_loss: 0.8402
Epoch [    9/   10] | d_loss: 1.3620 | g_loss: 0.8509
Epoch [    9/   10] | d_loss: 1.3333 | g_loss: 0.8515
Epoch [    9/   10] | d_loss: 1.3876 | g_loss: 0.8232
Epoch [    9/   10] | d_loss: 1.3801 | g_loss: 0.8145
Epoch [    9/   10] | d_loss: 1.3896 | g_loss: 0.7928
Epoch [    9/   10] | d_loss: 1.3775 | g_loss: 0.7981
Epoch [    9/   10] | d_loss: 1.3855 | g_loss: 0.7924
Epoch [    9/   10] | d_loss: 1.3891 | g_loss: 0.7990
Epoch [    9/   10] | d_loss: 1.3775 | g_loss: 0.8286
Epoch [    9/   10] | d_loss: 1.3741 | g_loss: 0.8287
Epoch [   10/   10] | d_loss: 1.3604 | g_loss: 0.8070
Epoch [   10/   10] | d_loss: 1.3788 | g_loss: 0.8017
Epoch [   10/   10] | d_loss: 1.3647 | g_loss: 0.8185
Epoch [   10/   10] | d_loss: 1.3553 | g_loss: 0.8040
Epoch [   10/   10] | d_loss: 1.3613 | g_loss: 0.8039
Epoch [   10/   10] | d_loss: 1.3671 | g_loss: 0.8125
Epoch [   10/   10] | d_loss: 1.3668 | g_loss: 0.8533
Epoch [   10/   10] | d_loss: 1.3563 | g_loss: 0.8160
Epoch [   10/   10] | d_loss: 1.3517 | g_loss: 0.8423
Epoch [   10/   10] | d_loss: 1.3023 | g_loss: 0.8799
Epoch [   10/   10] | d_loss: 1.3927 | g_loss: 0.9026
Epoch [   10/   10] | d_loss: 1.3624 | g_loss: 0.8073
Epoch [   10/   10] | d_loss: 1.3818 | g_loss: 0.8493
Epoch [   10/   10] | d_loss: 1.3656 | g_loss: 0.8050
Epoch [   10/   10] | d_loss: 1.3880 | g_loss: 0.8095
Epoch [   10/   10] | d_loss: 1.3739 | g_loss: 0.8076
Epoch [   10/   10] | d_loss: 1.3665 | g_loss: 0.7989
Epoch [   10/   10] | d_loss: 1.3734 | g_loss: 0.7947
Epoch [   10/   10] | d_loss: 1.3674 | g_loss: 0.7954
Epoch [   10/   10] | d_loss: 1.3688 | g_loss: 0.7916
Epoch [   10/   10] | d_loss: 1.3828 | g_loss: 0.7932
Epoch [   10/   10] | d_loss: 1.3657 | g_loss: 0.7940
Epoch [   10/   10] | d_loss: 1.3654 | g_loss: 0.7998
Epoch [   10/   10] | d_loss: 1.3714 | g_loss: 0.7878
Epoch [   10/   10] | d_loss: 1.3620 | g_loss: 0.7873
Epoch [   10/   10] | d_loss: 1.3826 | g_loss: 0.8003
Epoch [   10/   10] | d_loss: 1.3675 | g_loss: 0.8156
Epoch [   10/   10] | d_loss: 1.3741 | g_loss: 0.8622
Epoch [   10/   10] | d_loss: 1.3564 | g_loss: 0.8377
Epoch [   10/   10] | d_loss: 1.3510 | g_loss: 0.8286
Epoch [   10/   10] | d_loss: 1.3251 | g_loss: 0.8636
Epoch [   10/   10] | d_loss: 1.2989 | g_loss: 0.8453
Epoch [   10/   10] | d_loss: 1.3145 | g_loss: 0.8124
Epoch [   10/   10] | d_loss: 1.3722 | g_loss: 0.8257
Epoch [   10/   10] | d_loss: 1.3657 | g_loss: 0.7977
Epoch [   10/   10] | d_loss: 1.3834 | g_loss: 0.8130
Epoch [   10/   10] | d_loss: 1.3597 | g_loss: 0.8098
Epoch [   10/   10] | d_loss: 1.3490 | g_loss: 0.8081
Epoch [   10/   10] | d_loss: 1.3484 | g_loss: 0.8116
Epoch [   10/   10] | d_loss: 1.3518 | g_loss: 0.8060
Epoch [   10/   10] | d_loss: 1.3601 | g_loss: 0.8514
Epoch [   10/   10] | d_loss: 1.3550 | g_loss: 0.8159
Epoch [   10/   10] | d_loss: 1.3565 | g_loss: 0.8582
Epoch [   10/   10] | d_loss: 1.2981 | g_loss: 0.8632
Epoch [   10/   10] | d_loss: 1.3087 | g_loss: 0.8558
Epoch [   10/   10] | d_loss: 1.3597 | g_loss: 0.8044
Epoch [   10/   10] | d_loss: 1.3525 | g_loss: 0.8284
Epoch [   10/   10] | d_loss: 1.3357 | g_loss: 0.8713
Epoch [   10/   10] | d_loss: 1.3375 | g_loss: 0.8558
Epoch [   10/   10] | d_loss: 1.2997 | g_loss: 0.9120
Epoch [   10/   10] | d_loss: 1.3373 | g_loss: 0.8673
Epoch [   10/   10] | d_loss: 1.3782 | g_loss: 0.9199
Epoch [   10/   10] | d_loss: 1.3574 | g_loss: 0.8255
Epoch [   10/   10] | d_loss: 1.3240 | g_loss: 0.8602
Epoch [   10/   10] | d_loss: 1.3721 | g_loss: 0.8386
Epoch [   10/   10] | d_loss: 1.3726 | g_loss: 0.8611
Epoch [   10/   10] | d_loss: 1.3789 | g_loss: 0.8161
Training time: 1704.8 seconds

Training loss

Plot the training losses for the generator and discriminator, recorded after each epoch.

fig, ax = plt.subplots()
losses = np.array(losses)
plt.plot(losses.T[0], label='Discriminator', alpha=0.5)
plt.plot(losses.T[1], label='Generator', alpha=0.5)
plt.title("Training Losses")
plt.legend()

<matplotlib.legend.Legend at 0x1c20d87e850>

Generator samples from training

View samples of images from the generator, and answer a question about the strengths and weaknesses of your trained models.

# helper function for viewing a list of passed in sample images
def view_samples(epoch, samples):
    fig, axes = plt.subplots(figsize=(16,4), nrows=2, ncols=8, sharey=True, sharex=True)
    for ax, img in zip(axes.flatten(), samples[epoch]):
        img = img.detach().cpu().numpy()
        img = np.transpose(img, (1, 2, 0))
        img = ((img + 1)*255 / (2)).astype(np.uint8)
        ax.xaxis.set_visible(False)
        ax.yaxis.set_visible(False)
        im = ax.imshow(img.reshape((32,32,3)))

# Load samples from generator, taken while training
with open('train_samples.pkl', 'rb') as f:
    samples = pkl.load(f)

# epoch=1, lr=0.001, dim=512, z=128, batch=32
#_ = view_samples(-1, samples)

# epoch=1, lr=0.001, dim=128, z=128, batch=32
#_ = view_samples(-1, samples)

# epoch=10, lr=0.001, dim=128, z=128, batch=32 t=1710 seconds 
#_ = view_samples(-1, samples)

# epoch=50, lr=0.001, dim=128, z=128, batch=32, training time 1.5h
#_ = view_samples(-1, samples)

# decreasing learning rate
# epoch=10, lr=0.0002, dim=128, z=128, batch=32 t=1713 seconds 
#_ = view_samples(-1, samples)

# decreasing learning rate, increase number of epochs
# epoch=50, lr=0.0002, dim=128, z=128, batch=32
#_ = view_samples(-1, samples)

# dropout after each convolutional layer in discriminator
# epoch=10, lr=0.0002, dim=128, z=128, batch=32 t=1715 seconds
#_ = view_samples(-1, samples)

# no dropout after each convolutional layer in discriminator
# epoch=10, lr=0.0002, dim=128, z=128, batch=32 t=1715 seconds
#_ = view_samples(-1, samples)

# decreasing learning rate
# epoch=10, lr=0.0002, dim=128, z=128, batch=32 t=1703 seconds 
#_ = view_samples(-1, samples)

# epoch=50, lr=0.0002, dim=128, z=128, batch=32 t=8531 seconds 
#_ = view_samples(-1, samples)

# epoch=10, lr=0.0002, dim=128, z=128, batch=32 t=8531 seconds 
#_ = view_samples(-1, samples)

# epoch=10, lr=0.0002, dim=128, z=128, batch=32 t=8531 seconds 
#_ = view_samples(-1, samples)

Question: What do you notice about your generated samples and how might you improve this model?

When you answer this question, consider the following factors:

• The dataset is biased; it is made of "celebrity" faces that are mostly white
• Model size; larger models have the opportunity to learn more features in a data feature space
• Optimization strategy; optimizers and number of epochs affect your final result

Answer: (Write your answer in this cell) Data set is obviously biased and represents a unique subset of human faces. Apparently, this limitation allows to have relatively shallow generator and discriminator which have only three convolutional layers each. Increasing number of layers to capture more image features or dimension of layers to increase number of "filters" didn't result in a significant improvement of generated images. Changing learning rate from 0.001 to 0.0002 had a very significant impact on the output. Image quality at learning rate of 0.001 and training at 50 epoches was about the same as that images trained at 0.0002 learning rate and just 10 epochs - about 1/5 of training time of 50 epoches. Increasing number of epochs from 10 to 50 was decreasing discriminator loss and images were getting better. Interestingly, this increase was not leading to significant improvement. For example, [45] was trained on 10 epochs and [28] on 50. Another example of [43] and [88] on 10 epochs vs. [78] on 50. Model behaviour on 10 epochs was not stable. For example, [45] vs [73]. Apparently, increasing number of epochs, although very slowly, decreases loss whereas on smaller number of epochs loss may stay at slightly higher or slightly lower constant level. Likely improvement for this particular architecure wouldbe increase of number of epochs.

Submitting This Project

When submitting this project, make sure to run all the cells before saving the notebook. Save the notebook file as "dlnd_face_generation.ipynb" and save it as a HTML file under "File" -> "Download as". Include the "problem_unittests.py" files in your submission.